The need for efficient and effective methods of variable group selection in regression analysis is increasingly recognized. In many applications of data analytics, a natural grouping structure exists between variables. It is desirable to select variable groups, rather than individual variables, for better prediction and interpretability. For use in decision support, understanding the relationships between groups of variables is often more relevant. The following represents examples of relevant applications:
In infrastructure analytics, it is advantageous to group variables by functions, e.g. treat CPU related variables as a group, memory related variables as another group, etc.
In micro-array data analysis, genes belonging to the same functional group or to the same gene cluster may be grouped for more robust and meaningful analysis.
Of particular importance is the fact that group variable selection is a critical component of temporal causal modeling methods, in which it is desirable to discover causal/dependency relationship between time series rather than individual lagged variables.
Furthermore, group variable selection is also an important component in spatial temporal causal modeling, in which it is desired to discover causal/dependency relationship between spatial temporal features, assuming values at different points in space and time.
In these settings, selecting the right groups of variables is often more relevant to the subsequent use of estimated models, which may involve interpreting the models and making decisions based on them.
Recently, several methods have been proposed to address this variable group selection problem, in the context of linear regression (for example, Yuan, M., Lin, Y., Model selection and estimation in regression with grouped variables, J. R. Statist. Soc. B, 68, 4967, 2006). These methods are based on extending the Lasso formulation (Tibshirani, R., Regression shrinkage and selection via the lasso, J. Royal, Statist. Soc B., 58(1), 267-288, 1996) by modifying the regularization penalty term to account for the group structure. Hence, these existing methods do not adequately address the problem of variable group selection in presence of non-linearity in the data. The relatively high computational requirement of these existing methods has also been a hindrance to the application to real world problems having hundreds and thousands of variables.